Min tokens in token limiter #2400

giorgossideris · 2024-04-16T11:55:01Z

Why are these changes needed?

A min_tokens_threshold is added to MessageTokenLimiter, which gives the option to cut tokens only after the specified message limit is reached.

Related issue number

Closes #2306.

Checks

I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
I've added tests (if relevant) corresponding to the changes introduced in this PR.
I've made sure all auto checks have passed.

codecov-commenter · 2024-04-16T12:04:42Z

Codecov Report

Attention: Patch coverage is 0% with 17 lines in your changes are missing coverage. Please review.

Project coverage is 33.50%. Comparing base (ded2d61) to head (6b05ab3).
Report is 7 commits behind head on main.

Files	Patch %	Lines
...togen/agentchat/contrib/capabilities/transforms.py	0.00%	17 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2400      +/-   ##
==========================================
+ Coverage   33.33%   33.50%   +0.17%     
==========================================
  Files          83       81       -2     
  Lines        8636     8741     +105     
  Branches     1835     1868      +33     
==========================================
+ Hits         2879     2929      +50     
- Misses       5516     5564      +48     
- Partials      241      248       +7

Flag	Coverage Δ
unittests	`33.50% <0.00%> (+0.17%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

WaelKarkoub · 2024-04-16T13:33:21Z

@giorgossideris Thanks for the PR! min_tokens_threshold makes sense for MessageTokenLimiter. Like we discussed before, MessageHistoryLimiter shouldn't concern itself with the number of tokens to avoid overlap in responsibilities.

WaelKarkoub

PR is shaping up nicely

autogen/agentchat/contrib/capabilities/transforms.py

WaelKarkoub · 2024-04-17T19:06:04Z

Just a reminder to update the guide and notebook

autogen/agentchat/contrib/capabilities/transforms.py

Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com>

…orgossideris/autogen into min_tokens_in_history_limiter

giorgossideris · 2024-04-18T07:40:58Z

@WaelKarkoub I made the discussed changes, although there are two points that concern me:

About the min_tokens < max_tokens condition: what if the user wants to set min_tokens to the max token limit of the model?
(Not really related to this PR) Notebook and website docs are not aligned (same idea but different structure). I am not sure if this is something that we want to avoid.

WaelKarkoub

Thank you for the changes! A couple more looks and this is mergeable.

Your current validator makes sense. min_tokens <= max_tokens
No need for a major rewrite, just a simple demonstration of how min_tokens works is enough

notebook/agentchat_transform_messages.ipynb

website/docs/topics/long_contexts.md

autogen/agentchat/contrib/capabilities/transforms.py

giorgossideris · 2024-04-19T20:18:30Z

@WaelKarkoub I made the requested changes.

WaelKarkoub

Thanks, looks good!

gitguardian · 2024-04-22T14:03:57Z

⚠️ GitGuardian has uncovered 5 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	Secret	Commit	Filename
-	Generic High Entropy Secret	`899c472`	test/oai/test_utils.py	View secret
-	Generic CLI Secret	`df20e0b`	.github/workflows/dotnet-release.yml	View secret
-	Generic High Entropy Secret	`899c472`	test/oai/test_utils.py	View secret
-	Generic High Entropy Secret	`899c472`	test/oai/test_utils.py	View secret
-	Generic High Entropy Secret	`899c472`	test/oai/test_utils.py	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secrets safely. Learn here the best practices.
Revoke and rotate these secrets.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!}

giorgossideris · 2024-04-28T11:29:49Z

Hello @ekzhu, do you want me to do anything related to the failed test?

* Add minimum token threshold in MessageHistoryLimiter * Update transforms tests for the threshold * Move min_threshold_tokens from Message to Token Limiter * Optimize _check_tokens_threshold Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com> * Apply requested changes (renaming, phrasing, validations) * Fix format * Fix _check_tokens_threshold logic * Update docs and notebook * Improve phrasing * Add min_tokens example in notebook * Add min_tokens example in website docs * Add min_tokens example in notebook * Update website docs to be in sync with get_logs change --------- Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com> Co-authored-by: Chi Wang <wang.chi@microsoft.com>

giorgossideris added 2 commits April 16, 2024 14:11

Add minimum token threshold in MessageHistoryLimiter

3b8586f

Update transforms tests for the threshold

b55a68d

sonichi requested review from WaelKarkoub and gagb April 16, 2024 12:50

giorgossideris changed the title ~~Min tokens in history limiter~~ Min tokens in token limiter Apr 17, 2024

Move min_threshold_tokens from Message to Token Limiter

64737d6

WaelKarkoub suggested changes Apr 17, 2024

View reviewed changes

WaelKarkoub reviewed Apr 17, 2024

View reviewed changes

autogen/agentchat/contrib/capabilities/transforms.py Outdated Show resolved Hide resolved

giorgossideris and others added 6 commits April 18, 2024 09:30

Optimize _check_tokens_threshold

509f4bc

Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com>

Apply requested changes (renaming, phrasing, validations)

1997ad9

Merge branch 'min_tokens_in_history_limiter' of https://github.com/gi…

4d0759d

…orgossideris/autogen into min_tokens_in_history_limiter

Fix format

93b10d0

Fix _check_tokens_threshold logic

f3e1284

Update docs and notebook

73ff56a

giorgossideris requested a review from WaelKarkoub April 18, 2024 07:41

WaelKarkoub suggested changes Apr 18, 2024

View reviewed changes

giorgossideris added 4 commits April 19, 2024 22:35

Improve phrasing

763e888

Add min_tokens example in notebook

5437731

Add min_tokens example in website docs

fd38a5c

Add min_tokens example in notebook

d14e438

giorgossideris requested a review from WaelKarkoub April 19, 2024 20:18

gagb approved these changes Apr 20, 2024

View reviewed changes

giorgossideris and others added 2 commits April 20, 2024 15:52

Update website docs to be in sync with get_logs change

7b44d9f

Merge branch 'main' into min_tokens_in_history_limiter

899c472

WaelKarkoub approved these changes Apr 22, 2024

View reviewed changes

Merge branch 'main' into min_tokens_in_history_limiter

4cd0b01

WaelKarkoub requested a review from ekzhu April 23, 2024 17:35

ekzhu approved these changes Apr 24, 2024

View reviewed changes

WaelKarkoub and others added 3 commits April 24, 2024 18:14

Merge branch 'main' into min_tokens_in_history_limiter

6b05ab3

Merge branch 'main' into min_tokens_in_history_limiter

3d232de

Merge branch 'main' into min_tokens_in_history_limiter

df20e0b

Merge branch 'main' into min_tokens_in_history_limiter

3b949d4

sonichi added this pull request to the merge queue Apr 29, 2024

Merged via the queue into microsoft:main with commit 11a4342 Apr 29, 2024
68 of 82 checks passed

sonichi had a problem deploying to openai1 May 29, 2024 01:59 — with GitHub Actions Failure

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Min tokens in token limiter #2400

Min tokens in token limiter #2400

giorgossideris commented Apr 16, 2024 •

edited

Loading

codecov-commenter commented Apr 16, 2024 •

edited

Loading

WaelKarkoub commented Apr 16, 2024 •

edited

Loading

WaelKarkoub left a comment

WaelKarkoub commented Apr 17, 2024

giorgossideris commented Apr 18, 2024

WaelKarkoub left a comment •

edited

Loading

giorgossideris commented Apr 19, 2024

WaelKarkoub left a comment

gitguardian bot commented Apr 22, 2024 •

edited

Loading

giorgossideris commented Apr 28, 2024

Min tokens in token limiter #2400

Min tokens in token limiter #2400

Conversation

giorgossideris commented Apr 16, 2024 • edited Loading

Why are these changes needed?

Related issue number

Checks

codecov-commenter commented Apr 16, 2024 • edited Loading

Codecov Report

WaelKarkoub commented Apr 16, 2024 • edited Loading

WaelKarkoub left a comment

Choose a reason for hiding this comment

WaelKarkoub commented Apr 17, 2024

giorgossideris commented Apr 18, 2024

WaelKarkoub left a comment • edited Loading

Choose a reason for hiding this comment

giorgossideris commented Apr 19, 2024

WaelKarkoub left a comment

Choose a reason for hiding this comment

gitguardian bot commented Apr 22, 2024 • edited Loading

⚠️ GitGuardian has uncovered 5 secrets following the scan of your pull request.

giorgossideris commented Apr 28, 2024

giorgossideris commented Apr 16, 2024 •

edited

Loading

codecov-commenter commented Apr 16, 2024 •

edited

Loading

WaelKarkoub commented Apr 16, 2024 •

edited

Loading

WaelKarkoub left a comment •

edited

Loading

gitguardian bot commented Apr 22, 2024 •

edited

Loading